HW 02

Exploratory Data Analysis and Visualizations (STAT GR5702)

PART I

Problem 1: Occupational Mobility

Source: Chapter 4, p. 73 no. 5

According to R help page, the Yamaguchi87 dataset in vcdExtra has become a classic for models comparing two-way mobility tables. Note that this package contains a frequency data frame with 75 observations on the following 4 variables. The total sample size is 28,887.

library(ggplot2)
library(vcdExtra)
library(magrittr)
library(knitr)
library(dplyr)
library(tidyr)
# Open and read the data
data(Yamaguchi87)
y <<- data.frame(Yamaguchi87)

a) How do the distribution of occupations of the sons in the three countries compare?

According to the overall plot of the distribution of occupations of the sons below, the number of jobs are highest in US compared to UK or Japan in all occupation areas, including those categorized as Upper NonManuals (UpNM), Lower NonManuals (LoNM), Upper Manuals (UpM), Lower Manuals (LoNM), and Farming (Farm). The opportunity for the son’s jobs seems to be the highest in US, followed by UK then Japan.

By observing the distribution by occupations shown in the following paragraph, all categories but Farm actually shares very similar pattern, that US has the highest number of occupations, followed by UK, then Japan. Upper nonmanuals are professionals, managers, and officials; lower nonmanuals are proprietors, sales workers, and clerical workers; upper manuals are skilled workers; lower manuals are semiskilled and unskilled nonfarm workers; and farm workers are farmers and farm laborers. For the last category, US still has the highest number of jobs but UK comes to be the least among the three country.

# Distribution plot by occupation
ggplot(y,aes(x=Country,y=Freq,fill=Son)) + geom_bar(stat="identity") + 
  ggtitle("Distributions by Occupations of the Sons in US, UK, and Japan") + ylab("Frequency") + xlab("Country") +
  facet_grid( . ~ Son) +guides(fill=FALSE)

b) How do the distributions of the sons’ and fathers’ occupations in the UK compare?

To compare the occupation distributions between sons and fathers, I plotted a side-by-side frequency graph by occupation in an ascending order below. It follows that nonmanual works are more dominant among the sons (e.g. Sons have higher frequency for UpNM and LoNM), while the fathers have higher number of occupations in upper and lower manuals and farming jobs.

c) Are you surprised by the results or are they what you would have expected?

I am not quite surprised by the results. I would expect US to have the highest frequency on both the father and son occupations compared to UK or Japan. A reason for this is the population factor. I also expected a similar pattern for occupation distribution in UK across all job categories between the fathers and the sons. Younger generations tend to have better lives and more opportunities statistically, where most parents want to set out better paths for their children. It is definitely more common to encounter fathers with manual or farming job and sons with nonmanual labors.

---

Problem 2: Whisky

Source: Chapter 4, p. 73 no. 6 from Simmons Survey. The Whisky file contains a data frame with 2,218 observations on the following 21 variables. All variables are coded 1 if consumed in last year, 0 if not.

a) Draw a barchart of the number of respondents per brand. What ordering of the brands do you think is best?

The following graph shows the frequency of brands used in last year for the respondents who report consuming scotch. We see that scotch Chivas Regal was reported to be the most consumed, and Singleton was the least consumed.

In order to see the distribution more clearly, it would be best to sort the brand in the descending order by its frequency of use. In this case, Chivas Regal would be on the top and the Singleton would be at the very bottom of the graph.

b) There are 20 named brands and a further category Other.brands. Plot a chart containing big brands and grouping smaller brand names to “Other” brand. What cutoff would you use for defining a big brand?

According to the ordered graph in (a), we see that the frequency drops significantly after Glenfiddich brand. That is, from over 300 to a little over 100. This would therefore be a good cutoff point (e.g. 300) for grouping those with frequency less than that of Glenfiddich scotch to brand “Other”.

c) Two other dataset versions are called whiskey and whiskey_brands. How do you incorporate this information in your graphics? By using a different ordering, a colour, or drawing two graphics rather than one?

To incorporate this both whiskey and whiskey_brands in one graphic, I would add both Incidence and Freq and then plot it with different coloring and stack them on top of each other. After trying out different graphs, I chose to incorporate it this way because the graph looks informative and involves both ordering and colors. For instance, in the following graph we can see different types (Blend v. Single Malt) and origins of various whiskey brands (places as a legend).

The data set is from Simmons Study of Media and Markets and contains the incidence matrix for scotch brands used in last year for those households who report consuming scotch.

d) Which spellings, ‘whisky’ or ‘whiskey’ is more appropriate?

The spelling whisky is generally used for whiskies distilled in Scotland, Canada, Australia, Japan and Europe, while whiskey is used for the spirits distilled in Ireland and America. In this case, since many of these brands come from Scotland, it would be more appropriate to use whisky than whiskey.

#### Problem 3: Olive Oils from Italy Source: Chapter 5, p. 97 no. 8 Various fatty acid measurements. A data frame with 572 observations.

#####a) Draw a default parallel coordinate plot and describe various features you can see. From the graph below, you can see that there are 11 variables of the olive dataset, containing eicosenoic, arachidic, linolenic, linoleic, and so on. These features correlate to each other in within their region: noth, south or sardinia. ..

#####b) Draw the same plot and color the oils by the region they come from. What additional information can you find? Palmitic and palmitoleic are positively correlated to each other. Oleic is negatively correlated with palmitic, palmitoleic and linoleic.

#####c) Discuss which features of the dataset are easier to see with a PCP plot and which are easier to see with a scatterplot matrix? It is easier to see in scatterplot matrix than PCP, as we can see outliers and significance of each variable more clearly. Cluster groups and the diagonal matrices help identify better the correlation between one variable to another. It is a little harder to see using PCP since the ordering becomes more sensible. However, we see that there may be some subgroup of classes within PCP that would be an advantage over scatterplot.

Problem 4: Hertzsprung-Russell

Source: Chapter 5, p. 97 no. 10 Data for the Hertzsprung-Russell Diagram of the Star Cluster CYG OB1, which contains 47 stars in the direction of Cygnus, from C.Doom. The first variable is the logarithm of the effective temperature at the surface of the star (Te) and the second one is the logarithm of its light intencity (L/L0).
A data frame with 47 observations on the following 2 variables log.Te Logarithm of the effective temperature at the surface of the star (Te). log.light Logarithm of its light intencity (L/L0).

a) Plot Y against X. How does the plot differ from the plots you find on the web, for instance from a google search for images of this diagram?

The plot below shows that Hertzsprung-Russell diagrams plot star luminosity (brightness) against temperature (colour). The first one was drawn just over 100 years ago, based on dataset from the Yale Trigonometric Parallax.

HR Scatterplot resembles the graphic found on google better compared to that of StarsCYG. This makes sense because there are more datapoints shown in the diagram. In the Hertzsprung-Russell diagram, which is the scatterplot of these data points, where the log temperature is plotted from left to right, two groups of points are seen: the majority which tend to follow a steep band and four stars in the upper corner. In the astronomy the 43 stars are said to lie on the main sequence and the four remaining stars are called “giants” (the points 11, 20, 30, 34).

b) The plots seem to use different numbers of stars. Are some more likely to be used than others?

Yes. There are plots with less points showing the composition but there are also other plots with more points that show exact situations.

c) Any suggestions?

After reading a few notes from Chapter 13, I would want to base my coloring on the BV index. This way, the diagram would have more contrast, hues and saturation. I would also manipulate datapoints on the graph for clustering purposes, which would trigger identifying patterns and conclusion from the data (an example is the Super Giants cluster name).

---

Problem 5: Bodyfat

Source: Chapter 6, p. 130 no. 5 Dataset bodyfat provides estimates of the percentage of body fat of 252 men, determined by underwater weighing and body circumference measurements. Draw a parallel coordinate plot for the dataset.

a) Are there any outliers? What can you say about them?

Yes, there are certainly outliers. We can see from the plot that some lines do not follow the normal aggregated lines in the middle. For instace, some have points in 1.00 value when majority of lines are close to 0.25 for ankle category. We can also see from the following PCP plot that is ordered, some lines are off the flow, for instance the green lines in the upper part, which also confirm our claim about existing outliers.

b) Can you deduce anything about the height variable?

Specifically for the height variable, the range is wide, meaning that data ranges from the lowest to the highest value (from 0 to 1). It also suggests that those who are older with higher bodyfat tend to be shorter and those who are younger with low bodyfat tend to be taller. We can also derive from this conclusion and through the plot that the height variable is directly related to age.

c) What can you say about the relationship between the first two variables: density and bodyfat?

Looking at the following graph, the density variable is inversely proportional to the bodyfat variable, within the range of 0.0 and 1.0. In this case, when density is 1.0, bodyfat would be 0.0. To make sense of this we can think of bones that are very heavy, which would not have any fat.

d) Do you think the ordering of the variables is sensibile? What alternative orderings might be informative?

Yes, the ordering of variables is sensible. We can see from the above plot that when paired or compared side by side, two variables in PCP plots can show strong correlation, for instance in the case for density and bodyfat. An alternative orderings that might be informative is splitting it into age groups, as shown below. By flipping the axis and observing through different age groups, we can also see how bodyparts correlate to one another in their order. All age groups seem to share similar pattern.

---

Problem 6: Wine

Source: Chapter 6, p. 130 no. 7 The wine dataset contains data of three classes of wine. #####a) Use PCP to investigate how well the variables separate these classes. Variable data across classes are pretty consistent. Blue lines (class)

library(pgmm)
library(gclus)
library(rattle)

rm(list=ls())
data(wine)
# wine <- as.data.frame(wine)
ggparcoord(wine, columns = c(1:13), scale = "uniminmax",
           groupColumn = "class")+
  geom_line(alpha=0.21)+
  facet_wrap(~class, ncol = 1)+
  ggtitle("Wine Characteristic by Class")+
  theme(plot.title=element_text(hjust=0.5),
        panel.background = element_rect(fill = "lightgray"))+
  xlab("Wine Characteristic") + 
  ylab("Value")+
  coord_flip()

b) Are there any outliers?

Yes, by observing the graph above we see that there are some outliers within each class, Barbera, Barolo and Grignolino. However, data look very consistent and hence outliers would not affect the dataset as much. We see that data points in Grignolino has more outliers than Barbera or Barolo.

c) Is there evidence of subgroups within the classes?

We can perhaps say that there are subgroups within these classes. For instance, the very tight lines among different classes suggest their same exact patterns and behaviors. However, there are some lines that do not follow certain flow as the other (e.g. slopes are negative when a subclass shows an increasing line or positive slope), but they are not outliers as well because there are many of them that behave very similarly. In this case, we can call there are a few subgroups within the classes.

---

PART II

Choose two datasets from the above exercises. For each, provide insight, through graphs and text, on an issue that goes beyond those discussed in the homework exercises. Any type of graph is permitted, whether or not covered in class, so you may take this as an opportunity to explore new options.

cols <- c("orange", "red", "green")
cols <- c("orange", "red", "green")
ggplot(d,aes(x=arachidic,y=stearic, colour= Region, size=3))+geom_point()+
  facet_wrap(~Area)+ ggtitle("Scatterplot by Areas")+ theme(plot.title=element_text(hjust=0.5))

Warning in gzfile(file, "wb") :
  cannot open compressed file '/Users/Moorissa/.rstudio-desktop/notebooks/EB6F89F5-HW02-mmt2167/1/D3612B7D/c10gmxubuxe9n_t/5112663be6db46338aa8751b7d68caf1.snapshot', probable reason 'No such file or directory'
Error in gzfile(file, "wb") : cannot open the connection
Error in (function (which = dev.cur())  : 
  QuartzBitmap_Output - unable to open file '/Users/Moorissa/.rstudio-desktop/notebooks/EB6F89F5-HW02-mmt2167/1/D3612B7D/c10gmxubuxe9n_t/_rs_chunk_plot_001.png'

I always like stacked bar charts since it shows an overall sum of frequencies of different groups. Taking the first problem in another approach, I’d like to see the overall plot of the distribution of occupations of the sons above, the number of jobs are highest in US compared to UK or Japan in all occupation areas, including those categorized as Upper NonManuals (UpNM), Lower NonManuals (LoNM), Upper Manuals (UpM), Lower Manuals (LoNM), and Farming (Farm). This confirms our claim in the first problem that the opportunity for the son’s jobs seems to be the highest in US, followed by UK then Japan.